Expiring synchronized supplemental content in time-shifted media

ABSTRACT

Systems and methods are described for providing interactive content contextually related to an occurrence. An illustrative method generates for display, at a media consumption device, a display of the live event, wherein the display of the live event comprises the occurrence, determines a beginning of the occurrence in the display of the live event, in response to determining the beginning of the occurrence in the display of the live event, generates for simultaneous display, with the display of the live event, interactive content related to the occurrence, determines whether the occurrence in the live event has ended in real time, and in response to determining that the occurrence in the live event has ended in real time, ceases the generating for display of the interactive content related to the occurrence.

BACKGROUND

The present disclosure relates to expiring synchronized supplementalcontent in time-shifted media, and more particularly, to systems andmethods for preventing display of supplemental content after the end ofan occurrence related to the supplemental content.

SUMMARY

Advancements in digital transmission of media content have increased theamount of media content that can be made available to an audience. Forexample, a digital transmission of media content may include varioussupplemental content, such as enhanced and/or interactive content,related to the media content. Such supplemental content enhances theuser experience and allows the users to interact with the media content.Time-shifting technologies, which enable users to delay playback of alive transmission of media content, have also greatly enhanced the userexperience by allowing users to consume the media content andsupplemental content at a time of their own choosing (e.g., at a timeafter the live transmission). But such technologies may causetime-sensitive supplemental content (e.g., supplemental contentsynchronized with particular portions of media content) to becomeoutdated (e.g., no longer available, no longer accurate, etc.) by thetime the users consume a related portion of the media content. Also,even without user-initiated time shifts, these live events typicallyhave a delay between a real-time occurrence and its transmission, whichcan adversely affect supplemental content.

Accordingly, to overcome such problems, systems and methods aredisclosed herein for preventing output of supplemental contentsynchronized to an occurrence in the media content after the occurrencehas ended. In one example embodiment, which will be described furtherbelow with reference to the drawings, the media content is a video of alive football game that is being played back on a user's smartphone. Thefootball game includes a field goal attempt (i.e., an occurrence). Inthe example embodiment, the video is played back at a delay of 30seconds behind the real-life football game. Thus, what the user sees inthe video happened 30 seconds earlier in the real-life football game.When the field goal attempt is displayed in the video, a poll (i.e.,supplemental content) asking the user about the predicted outcome of thefield goal is displayed on the smartphone simultaneously with the video.The poll is relevant only to users who do not yet know the outcome ofthe field goal attempt. In that regard, the poll is displayed only fromthe time that the depiction of the field goal attempt in the video isdisplayed (i.e., 30 seconds after the field goal attempt in thereal-life football game began), until the time that the field goalattempt in the real-life football game ends (i.e., 30 seconds before thedepiction of the field goal attempt in the video ends). Thus, to theuser watching the video of the football game on the smartphone, the pollwill end before the field goal attempt is completed because the user iswatching the video of the football game on a 30-second-behind-real-timedelay. At the same time, display of the poll should not begin before theuser reaches the beginning of the field goal attempt in the delayedplayback of the media content, or the user experience may be spoiled bydisplay of a poll related to a field goal that the user does not yetknow is about to happen.

Improvements are thus required to enable supplemental content related tolive events to be provided during a period of time beginning when theoccurrence (e.g., the field goal) is displayed in the delayed playbackof the media content that the user is consuming, and ending when theoccurrence ends in real time. To that effect, the present disclosuredescribes systems and methods for determining when an occurrence beginsin a delayed playback of media content. For example, the describedsystems and methods may retrieve metadata associated with the mediacontent and identify, in the metadata, an indication of the beginning ofthe occurrence in the media content. Alternatively, the systems andmethods may determine an amount of delay between the live event and thedelayed playback of the media content by determining a time when acurrently displayed portion of the media content happened in real time,and comparing that time to the current time. Additionally oralternatively, the amount of delay may be determined by calculating anamount of time-shifting applied to the media content, such as by useractions (e.g., pausing, rewinding, etc.) affecting playback of the mediacontent. The systems and methods may then calculate an offset start timeof the occurrence by adding the amount of delay to the start time of theoccurrence in real time. In some instances, the systems and methods maydetermine the beginning of the occurrence in the media content byperforming audio or image processing on the media content to identifycharacteristics (e.g., graphics, sounds (such as cheering or vocalmatching), etc.) indicative of the beginning of the occurrence.

Once the beginning of the occurrence in the delayed playback of themedia content is determined, the described systems and methods generate,for simultaneous display with the media content, the supplementalcontent related to the occurrence. For example, the systems and methodsmay retrieve information related to the occurrence and generate a pollbased on the information. The poll may include a question regarding theoutcome of the occurrence. In some instances, the supplemental contentmay be displayed on a second device (i.e., a different device from thedevice on which the media content is played back), and the second devicemay receive a user selection of a selectable option (e.g., a pollresponse) included in the supplemental content. The second device maythen transmit an indication of the user selection to a remote device,which may receive indications of user selections from a plurality ofdevices and aggregate the responses to determine a result of the poll.

The present disclosure further describes systems and methods fordetermining when an occurrence ends in real time. For example, thedisclosed systems and methods may retrieve metadata associated with themedia content and identify, in the metadata, an indication of the end ofthe occurrence in real time. Once it is determined that the occurrencehas ended in real time, the systems and methods may cease generating fordisplay the supplemental content related to the occurrence in order toprevent the supplemental content from being displayed after theoccurrence has ended in real time. After the occurrence has ended inreal time and the supplemental content is no longer displayed, thesystems and methods may further determine, using the same or similarmethods to those described above for determining the beginning of theoccurrence in the media content, whether the occurrence has ended in thedelayed playback of the media content. Once the occurrence has ended inthe delayed playback of the media content (i.e., once the user has seenthe outcome of the field goal attempt), the systems and methods maygenerate for simultaneous display with the media content, additionalsupplemental content (e.g., the result of the poll).

In another example embodiment, the media content is a video of a realitytelevision show in which the audience (e.g., the viewers at home) areasked to cast a vote. The voting period starts at a particular point inthe show, and ends at midnight two days later. In the exampleembodiment, a user is watching the show on the night after it was firstbroadcast. Thus, when the user is watching the show, the voting periodhas already begun but is not yet over. The supplemental content includesthe names and pictures of three contestants in the show for whom theuser can cast a vote, an option to enter the user's choice, and anoption to submit the vote. In this example, the supplemental content isdisplayed only after the user reaches the particular point in thedelayed playback of the show when the voting is announced, and only ifthe particular point in the show is played back before the voting periodends (i.e., if the point in the show where the voting period isannounced is played back before midnight on the day that is two daysafter the show was first broadcast). That is, if the show is played backafter the voting period ends (i.e., after midnight on the day that istwo days after the show was first broadcast), the supplemental contentallowing the user to cast a vote will not be displayed. Instead,different supplemental content, such as a notice that the voting periodhas already ended, may be displayed in place of the supplemental contentallowing the user to cast a vote.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows an illustrative example of a user interface for providingsupplemental content related to an occurrence during a live event, inaccordance with some embodiments of the disclosure;

FIG. 2 shows an illustrative media system for providing supplementalcontent to different user devices, in accordance with some embodimentsof the disclosure;

FIG. 3 shows a diagram of illustrative media devices, in accordance withsome embodiments of the disclosure;

FIG. 4 shows a diagram of illustrative timelines for providingsupplemental content related to an occurrence during a live event, inaccordance with some embodiments of the disclosure;

FIG. 5 is a flowchart of an illustrative process for providingsupplemental content related to an occurrence during a live event, inaccordance with some embodiments of the disclosure;

FIG. 6 is a flowchart of a detailed illustrative process for determininga real-time ending of an occurrence in a live event, in accordance withsome embodiments of the disclosure;

FIG. 7 is a flowchart of a detailed illustrative process for determininga beginning of an occurrence in a display of a live event, in accordancewith some embodiments of the disclosure;

FIG. 8 is a flowchart of a detailed illustrative process for determiningan amount of delay between a live event and a display of a live event,in accordance with some embodiments of the disclosure;

FIG. 9 is a flowchart of another detailed illustrative process fordetermining an amount of delay between a live event and a display of alive event, in accordance with some embodiments of the disclosure;

FIG. 10 is a flowchart of a detailed illustrative process fordetermining a beginning of an occurrence in a display of a live event,in accordance with some embodiments of the disclosure;

FIG. 11 is a flowchart of another detailed illustrative process fordetermining a beginning of an occurrence in a display of a live event,in accordance with some embodiments of the disclosure; and

FIG. 12 is a flowchart of a detailed illustrative process for generatingfor simultaneous display, with a display of a live event, supplementalcontent related to an occurrence, in accordance with some embodiments ofthe disclosure.

DETAILED DESCRIPTION

FIG. 1 shows an illustrative example of a user interface for providingsupplemental content related to an occurrence during a live event, inaccordance with some embodiments of the disclosure. In particular, FIG.1 shows a scenario 100 where a media consumption device 102 displaysmedia content 104 including an occurrence during a live event. The mediaconsumption device 102 may be any device for accessing the contentdescribed below, such as a television, a Smart TV, a set-top box, anintegrated receiver decoder (IRD) for handling satellite television, adigital storage device, a digital media receiver (DMR), a digital mediaadapter (DMA), a streaming media device, a personal computer (PC), alaptop computer, a tablet computer, a WebTV box, a personal computertelevision (PC/TV), a PC media server, a PC media center, a handheldcomputer, a personal digital assistant (PDA), a portable video player, aportable music player, a portable gaming machine, a smartphone, awearable computer, an augmented and/or virtual reality display device,or any other television equipment, computing equipment, or wirelessdevice, a speaker or other audio output device, and/or any other displaydevice or audio output device usable to consume the media content 104 orcombination of the same.

In that regard, the media content may include an electronicallyconsumable media asset, such as include video, audio, still images,virtual and/or augmented reality renderings, television programming, aswell as pay-per-view programs, Internet content (e.g., streamingcontent, downloadable content, Webcasts, etc.), chat sessions, socialmedia, applications, games, and/or any other media or multimedia and/orcombination of the same and/or any type, form, or combination of mediavia which a live event may be presented. The media content 104 may be abroadcast, a stream, a download, and/or any other transmission of amedia asset from a content source to the media consumption device 102.

The media consumption device 102 further displays supplemental content110 including a prompt 112 and a plurality of selectable options 114 a,114 b, and 114 c. In the example shown in FIG. 1 , the media content 104shows a football game and the supplemental content 110 is a poll. Theprompt 112 is a question, and the selectable options 114 a, 114 b, and114 c include responses to the poll (e.g., radio buttons correspondingto “Yes” and “No” answers) and a “Submit” button. Those skilled in theart will appreciate that the supplemental content 110 can take variousforms and include various features and/or functionalities in addition toor instead of the example shown in FIG. 1 . For example, thesupplemental content may be information or statistics related to theoccurrence depicted in the media content 104; highlights, replays,augmented graphics or audio features related to the occurrence depictedin the media content 104; questionnaires, voting options, pollingoptions, etc.; and/or any other supplemental interactive or augmentedcontent related to the occurrence depicted in the media content 104.

FIG. 2 shows an illustrative media system for providing supplementalcontent to different user devices. In particular, FIG. 2 shows a system200 where the media consumption device 102 displays the media content104 received from a content source 206 via a communication link 208. Thecontent source 206 may be any server, broadcasting, or transmissionequipment by means of which a content provider provides the mediacontent 104 and/or supplemental content 110 to the media consumptiondevice 102 and/or a second device 204. In that regard, the communicationlink 208 may be any network or communication equipment or medium bymeans of which the media content 104 can be provided by the contentsource 206 to the media consumption device 102. In one illustrativeexample, the communication link 208 includes devices connected via theInternet.

In system 200, the supplemental content 110 is displayed on the seconddevice 204. The second device 204 can be any television, desktopcomputer, laptop computer, tablet computer, smartphone, wearablecomputer, augmented reality display device, speaker or other audiooutput device, and/or any other display device or audio output deviceusable to consume and/or interact with the supplemental content 110.Similar to the media content 104, the supplemental content 110 isprovided by the content source 206 to the second device 204 by means ofthe communication link 208.

A content generation application may be implemented on any one or acombination of the media consumption device 102, the second device 204,the content source 206, and/or a processing server 216, each of whichmay function as a stand-alone device or may be part of a network ofdevices. Various network configurations of devices may be implementedand are discussed in more detail below. In the system 200, there may bemultiple media devices, but only one of each is shown in FIG. 2 to avoidovercomplicating the drawing. In addition, each user may utilize morethan one type of media device and also more than one of each type ofmedia device.

The media consumption device 102 and second device 204 may be coupled toa communication network 209. The communication network 209 may be one ormore networks including the Internet, a mobile phone network, mobilevoice or data network (e.g., a 4G or LTE network), cable network, publicswitched telephone network, or other types of communications network orcombinations of communications networks. The content source 206, theprocessing server 216, the media consumption device 102, and the seconddevice 204 may be connected to the communication network 209 via one ormore communications paths 208, such as, a satellite path, a fiber-opticpath, a cable path, a path that supports Internet communications (e.g.,IPTV), free-space connections (e.g., for broadcast or other wirelesssignals), or any other suitable wired or wireless communications path orcombination of such paths.

Although communications paths 208 are not drawn directly between themedia consumption device 102, the second device 204, and the contentsource 206 or the processing server 216, these devices may communicatedirectly with each other via communication paths 208, such asshort-range point-to-point communication paths, such as USB cables, IEEE1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x,etc.), or other short-range communication via wired or wireless paths.BLUETOOTH is a certification mark owned by Bluetooth SIG, INC. Thedevices may also communicate with each other directly through anindirect path via the communication network 209.

The system 200 may include more than one content source 206, but onlyone is shown in FIG. 2 to avoid overcomplicating the drawing. Thecontent source 206 may include one or more types of content distributionequipment including television distribution facilities, cable systemheadends, satellite distribution facilities, programming sources (e.g.,television broadcasters, etc.), intermediate distribution facilitiesand/or servers, Internet providers, on-demand media servers, and othercontent providers. The content source 206 may be the originator ofcontent (e.g., a television broadcaster, a Webcast provider, etc.) ormay not be the originator of content (e.g., an on-demand contentprovider, an Internet provider of content of broadcast programs fordownloading, etc.). The content source 206 may include cable sources,satellite providers, on-demand providers, Internet providers,over-the-top content providers, or other providers of content. Thecontent source 206 may also include a remote media server used to storedifferent types of content (including video content selected by a user),in a location remote from any of the media devices. Systems and methodsfor remote storage of content, and providing remotely stored content touser equipment, are discussed in greater detail in connection with Elliset al., U.S. Pat. No. 7,761,892, issued Jul. 20, 2010, which is herebyincorporated by reference herein in its entirety. The content source 206may also provide supplemental content, as described above and below,and/or metadata associated with the media content and supplementalcontent.

Metadata may include program information, supplemental contentsynchronization information, start time indications, end timeindications, user preferences, user profile information, media listings,media-related information (e.g., broadcast times, broadcast channels,titles, descriptions, genre or category information, cast or playerinformation, websites, and any other type of data that enables acomputer to determine a start time and/or an end time of an occurrencedepicted in the media content.

The content generation applications may be, for example, stand-aloneapplications implemented on the media devices described above. Forexample, the content generation application may be implemented assoftware or a set of executable instructions which may be stored instorage 308 (described below with reference to FIG. 3 ), and executed bycontrol circuitry 304 (described below with reference to FIG. 3 ) of themedia consumption device 102. In some embodiments, content generationapplications are client/server applications where only a clientapplication resides on the media device, and a server applicationresides on the processing server 216. For example, a content generationapplication may be implemented partially as a client application on thecontrol circuitry 304 of the media device 102 and partially on theprocessing server 216 as a server application running on the controlcircuitry 304 of the processing server 216. When executed by the controlcircuitry 304 of the processing server 216, the content generationapplication may instruct the control circuitry 304 to generate thecontent generation application output (e.g., the supplemental content110) and transmit the generated output to the media consumption device102 and/or the second device 204. The server application may instructthe control circuitry 304 of the content source 216 to transmit metadatafor synchronization of the supplemental content 110 and the mediacontent 104 for display on media consumption device 102 and/or thesecond device 204. The client application may instruct the controlcircuitry 304 of the receiving media consumption device 102 and/or thesecond device 204 to generate the content generation application output(e.g., the supplemental content 110).

The media consumption device 102 and/or the second device 204 mayoperate in a cloud computing environment to access cloud services. In acloud computing environment, various types of computing services forcontent sharing, storage or distribution (e.g., informational sites orsocial networking sites) are provided by a collection ofnetwork-accessible computing and storage resources, referred to as “thecloud.” Cloud resources may be accessed by the media consumption device102 and/or the second device 204 using, for example, a web browser, adesktop application, a mobile application, and/or any combination ofaccess applications of the same. The media consumption device 102 and/orthe second device 204 may be a cloud client that relies on cloudcomputing for application delivery, or the media consumption device 102and/or the second device 204 may have some functionality without accessto cloud resources. For example, some applications running on the mediaconsumption device 102 may be cloud applications, that is, applicationsdelivered as a service over the Internet, while other applications maybe stored and run on the media consumption device 102. In someembodiments, the media consumption device 102 receives content frommultiple cloud resources simultaneously. For example, the mediaconsumption device 102 can stream video from one cloud resource whiledownloading supplemental content 110 from a second cloud resource. Orthe media consumption device 102 can download media content 104 andsupplemental content 110 from multiple cloud resources for moreefficient downloading. In some embodiments, the media consumption device102 uses cloud resources for processing operations, such as theprocessing operations performed by the processing circuitry 304described in relation to FIG. 3 .

FIG. 3 shows a generalized embodiment of illustrative media devices 300and 301. As depicted, the media device 300 may be a smartphone ortablet, whereas the media device 301 may be a home media system thatincludes an equipment device 316 (e.g., a set-top box, CPU, videogameconsole, etc.) powered by a processor 324. The media devices 300 and 301may receive content and data via an input/output (hereinafter “I/O”)path 302. The I/O path 302 may provide media content (e.g., broadcastprogramming, on-demand programming, Internet content, content availableover a local area network (LAN) or wide area network (WAN), and/or othercontent), supplemental content and data to control circuitry 304, whichincludes processing circuitry 306 and storage 308. The control circuitry304 may be used to send and receive commands, requests, and othersuitable data using the I/O path 302. The I/O path 302 may connect thecontrol circuitry 304 (and specifically the processing circuitry 306) toone or more communications paths (described below). I/O functions may beprovided by one or more of these communications paths, but are shown asa single path in FIG. 3 to avoid overcomplicating the drawing.

The control circuitry 304 may be based on any suitable processingcircuitry such as the processing circuitry 306. As referred to herein,processing circuitry should be understood to mean circuitry based on oneor more microprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry is distributed across multiple separate processorsor processing units, for example, multiple of the same type ofprocessing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). In some embodiments, the control circuitry 304 executesinstructions for a content generation application stored in memory(i.e., the storage 308). Specifically, the control circuitry 304 may beinstructed by the content generation application to perform thefunctions discussed above and below.

In client/server-based embodiments, the control circuitry 304 includescommunications circuitry suitable for communicating with a contentgeneration application server or other networks or servers. Theinstructions for carrying out the above-mentioned functionality may bestored on a server. Communications circuitry may include a cable modem,an integrated services digital network (ISDN) modem, a digitalsubscriber line (DSL) modem, a telephone modem, Ethernet card, or awireless modem for communications with other equipment, or any othersuitable communications circuitry. Such communications may involve theInternet or any other suitable communications networks or paths. Inaddition, communications circuitry may include circuitry that enablespeer-to-peer communication of media devices, or communication of mediadevices in locations remote from each other.

Memory may be an electronic storage device provided as the storage 308that is part of the control circuitry 304. As referred to herein, thephrase “electronic storage device” or “storage device” should beunderstood to mean any device for storing electronic data, computersoftware, or firmware, such as random-access memory, hard drives,optical drives, solid state devices, quantum storage devices, gamingconsoles, gaming media, or any other suitable fixed or removable storagedevices, and/or any combination of the same. Nonvolatile memory may alsobe used (e.g., to launch a boot-up routine and other instructions).Cloud-based storage, described in relation to FIG. 2 , may be used tosupplement the storage 308 or instead of the storage 308.

The control circuitry 304 may include audio-generating circuitry andtuning circuitry, such as one or more analog tuners, one or more MP3decoders or other digital decoding circuitry, or any other suitabletuning or audio circuits or combinations of such circuits. Encodingcircuitry (e.g., for converting over-the-air, analog, or digital signalsto audio signals for storage) may also be provided. The controlcircuitry 304 may also include scaler circuitry for upconverting anddownconverting content into the preferred output format of the mediadevice 300. The control circuitry 304 may also include digital-to-analogconverter circuitry and analog-to-digital converter circuitry forconverting between digital and analog signals. The tuning and encodingcircuitry may be used by the media device 300 to receive and to display,to play, or to record content. The circuitry described herein, includingfor example, the tuning, audio generating, encoding, decoding,encrypting, decrypting, scaler, and analog/digital circuitry, may beimplemented using software running on one or more general purpose orspecialized processors. Multiple tuners may be provided to handlesimultaneous tuning functions. If the storage 308 is provided as aseparate device from the media device 300, the tuning and encodingcircuitry (including multiple tuners) may be associated with the storage308.

A user may send instructions to the control circuitry 304 using a userinput interface 310 of the media device 300 or a user input interface320 of the media device 301. The user input interface 310 and the userinput interface 320 may be any suitable user interface, such as a remotecontrol, mouse, trackball, keypad, keyboard, touch screen, touchpad,stylus input, joystick, voice recognition interface, or other user inputinterfaces. Display 312 may be a touchscreen or touch-sensitive display.In such circumstances, user input interface 310 may be integrated withor combined with the display 312. Display 322 may be provided as astand-alone device or integrated with other elements of the media device301. Speakers 314 may be provided as integrated with other elements ofthe media device 300. In the case of the media device 301, speakers 318may be stand-alone units (e.g., smart speakers). The audio component ofvideos and other content displayed on the display 322 may be playedthrough the speakers 318. In some embodiments, the audio is distributedto a receiver (not shown), which processes and outputs the audio via thespeakers 318.

The content generation application may be implemented using any suitablearchitecture. For example, it may be a stand-alone application whollyimplemented on the media device 300. The content generation applicationand/or any instructions for performing any of the embodiments discussedherein may be encoded on computer readable media. Computer-readablemedia includes any media capable of storing data. In some embodiments,the content generation application is a client/server-based application.Data for use by a thick or thin client implemented on the media device300 or the media device 301 is retrieved on-demand by issuing requeststo a server remote to the media device 300 or the media device 301,respectively. For example, the media device 300 may receive inputs fromthe user via the input interface 310 and transmit those inputs to theremote server for processing and generating the corresponding outputs.The generated output is then transmitted to the media device 300 forpresentation to the user. Those skilled in the art will appreciate thatmedia consumption device 102 of FIG. 1 and second device 204 of FIG. 2may be implemented as media device 300 and/or media device 301 of FIG. 3.

FIG. 4 shows a diagram of illustrative timelines for providingsupplemental content related to an occurrence during a live event, inaccordance with some embodiments of the disclosure. In particular, FIG.4 shows a timeline during which the supplemental content 110 isdisplayed in relation to a timeline during which the occurrence to whichthe supplemental content 110 is related happened. As shown in FIG. 4 , adiagram 400 includes a live timeline 410 indicating relative times whenan occurrence in a live event starts 412 and ends 414. The diagram 400further includes a displayed timeline 420 indicating relative times whenthe occurrence in the live event is displayed in the media content 104depicting the occurrence in the live event, including the times when theoccurrence, as depicted in the media content 104, starts 422 and ends424. Additionally, the diagram 400 shows the period 426 during which thesupplemental content 110 is displayed.

In the example shown in FIG. 4 , an occurrence in a live event starts attime 412. Media content, such as the media content 104, may be generatedfor display by the content source 206, the processing server 216, and/orthe media consumption device 102. The media content 104 may be displayedon the media consumption device 102 in such a manner that, due totime-shifting caused by one or more of a processing delay, atransmission delay, and a user action affecting playback, a currentlydisplayed portion of the live event as depicted in the media content 104is delayed relative to the live event in real time. A content generationapplication executing on the content source 206, the processing server216, and/or the media consumption device 102, then determines whetherdisplay of the occurrence has begun in the delayed playback of the mediacontent 104. For example, the content generation application mayretrieve metadata associated with the media content 104 and identify, inthe metadata, an indication (e.g., a timestamp or other synchronizationdata) of the time 422 when display of a field goal attempt in a footballgame begins in the delayed playback of the media content 104.

The content generation application may further determine a start time ofthe occurrence in the live event in real time (i.e., the time 412). Forexample, the content generation application may retrieve metadataassociated with the media content 104 and identify, in the metadata, anindication (e.g., a timestamp or other data) of the time 412 when thefield goal attempt began in real time. The content generationapplication may then calculate an offset start time by adding the amountof delay between the live event in real time and depiction of the liveevent in the delayed playback of the media content 104, to the starttime of the occurrence in the live event in real time. For example, ifthe field goal attempt began at 2:34.15 PM in real time, and the delayis 30 seconds, the field goal will begin at 2:34.45 PM in the delayedplayback of the media content 104.

In yet another example, the content generation application may determinethe beginning of the occurrence as depicted in the media content 104(i.e., the time 422) by performing audio and/or image processing on themedia content 104 to identify graphics or audio included in the mediacontent 104 that indicate the beginning of the occurrence. In anembodiment, the content generation application applies patternrecognition and/or character recognition techniques to the media content104 to identify text, graphics, objects, and/or actions performed byobjects in the media content 104 that indicate the beginning of theoccurrence. For example, the content generation application may identifya gesture or sign displayed by a referee of a football game indicatingthat a field goal attempt is about to commence. The content generationapplication may also apply audio signature analysis and matchingtechniques to audio included in the media content 104 to identify wordsindicating the beginning of the occurrence, or identify sounds (e.g.,cheers, whistles, etc.) indicating the beginning of the occurrence.

While the supplemental content 110 is being generated for display, thecontent generation application determines whether the occurrence in thelive event has ended in real time (i.e., the content generationapplication determines whether the time 414 has passed). For example,the content generation application may determine whether the field goalattempt has been completed in the real-life football game. In someembodiments, the content generation application retrieves metadataassociated with the media content 104 and identifies, in the metadata,an indication (e.g., a timestamp or other synchronization data) of thetime 414 when the occurrence ends in real time. In another embodiment,the content generation application determines the end of the occurrencein real time (i.e., the time 414) by performing audio and/or imageprocessing on the media content 104 to identify graphics or audioincluded in the media content 104 that indicate the end of theoccurrence. For example, the content generation application may applypattern recognition and/or character recognition techniques to the mediacontent 104 to identify text, graphics, objects, and/or actionsperformed by objects in the media content 104 that indicate the end ofthe occurrence. For example, the content generation application mayidentify a gesture or sign displayed by a referee of a football gameindicating that a field goal attempt is complete. The content generationapplication may also apply audio signature analysis and matchingtechniques to audio included in the media content 104 to identify wordsindicating the end of the occurrence, or identify sounds (e.g., cheers,whistles, etc.) indicating the end of the occurrence.

Once the content generation application determines that the occurrencehas ended in real time (i.e., that the time 414 has passed), the contentgeneration application ceases generating for display the supplementalcontent 110. For example, the content generation application may stoptransmitting the supplemental content 110 and/or send a signal orcommand, to the media consumption device 102 or the second device 204,to stop displaying or generating for display the supplemental content110.

The content generation application may then determine whether theoccurrence has ended in the delayed playback of the media content 104(i.e., determine whether the time 424 has passed). The determiningwhether the occurrence has ended in the delayed playback of the mediacontent 104 may be performed by similar processes as described above fordetermining the beginning of the occurrence in the delayed playback ofthe media content 104, and those processes are not repeated here forpurposes of brevity.

Once the content generation application determines that the occurrencehas ended in the delayed playback of the media content 104 (i.e., thatthe time 414 has passed), the content generation application maygenerate for display the additional supplemental content 110 related tothe occurrence (e.g., the result of the poll).

FIG. 5 is a flowchart of an illustrative process for providingsupplemental content related to an occurrence during a live event, inaccordance with some embodiments of the disclosure. A process 500 forproviding supplemental content related to an occurrence during a liveevent may begin at block 502, where control circuitry, such as thecontrol circuitry 304 described above, generates for display a displayof a live event comprising an occurrence. The display of the live eventmay be media content, such as the media content 104 described above, andmay be displayed on a media device, such as the media consumption device102 described above. The live event may be an event happening in realtime that is being captured, recorded, and/or at least transientlystored for distribution, such as via broadcast, streaming, or othertransmission, as described above. For example, the live event may be afootball game, and the occurrence in the live event may be a field goalattempt, as described above.

At block 504, the control circuitry 304 determines a beginning of theoccurrence in the display of the live event. For example, the controlcircuitry 304 may determine whether display of the field goal attempthas begun in the playback of the media content 104. Subprocessesdescribing further details on how the control circuitry 304 determinesthe beginning of the occurrence in the display of the live event aredescribed below with reference to FIGS. 7, 10, and 11 .

At block 506, the control circuitry 304 retrieves information related tothe occurrence. For example, the control circuitry 304 may retrieve thenames and statistics of the players, and the teams for which they play,involved in the field goal attempt. The information may be stored in adatabase included in the content source 206 and/or other sources ofinformation accessible via the communication network 209.

At block 508, the control circuitry 304 generates for simultaneousdisplay, with the display of the live event, supplemental content, suchas the supplemental content 110, related to the occurrence. Thesupplemental content 110 may be interactive content such as a poll,enhanced content such as statistics or other information related to theoccurrence, and/or any of the other supplemental content 110 describedabove. In an embodiment where the supplemental content 110 is a poll,the control circuitry 304 generates the poll based on the informationretrieved at block 506. The supplemental content 110 may be displayedoverlaid onto the media content 104, and/or in such a way as not toobscure the media content 104. Alternatively, the supplemental content110 may be displayed on a second device 204, as described above andbelow.

At block 510, the control circuitry 304 receives a selection of aselectable option included in the supplemental content 110. For example,as noted above, the supplemental content 110 may include selectableoptions 114 a, 114 b, and 114 c, and the control circuitry 304 mayreceive an indication of a user selection of one or more of theselectable options 114 a, 114 b, and/or 114 c.

At block 512, the control circuitry 304 determines whether theoccurrence in the live event has ended in real time. For example, thecontrol circuitry 304 may determine whether the field goal attempt hasended in the football game in real time. Subprocesses describing furtherdetails on how the control circuitry 304 determines whether theoccurrence in the live event has ended in real time are described belowwith reference to FIG. 6 . In response to determining that theoccurrence in the live event has not ended in real time, process 500repeats block 512. In response to determining that the occurrence in thelive event has ended in real time, process 500 proceeds to block 514.

At block 514, the control circuitry 304 ceases generating for displaythe supplemental content 110 related to the occurrence. For example, thecontrol circuitry 304 may stop transmitting the supplemental content 110and/or send a signal or command to the media consumption device 102 orthe second device 204 to stop displaying or generating for display thesupplemental content 110.

At block 516, the control circuitry 304 determines whether theoccurrence in the live event has ended in the display of the live event.For example, the control circuitry 304 may determine whether display ofthe field goal attempt in the football game has ended in the delayedplayback of the media content 104. The control circuitry 304 maydetermine whether the occurrence in the live event has ended in thedisplay of the live event by similar processes to those described aboveand below for determining the beginning of the occurrence in the displayof the live event. In response to determining that the occurrence in thelive event has not ended in the display of the live event, process 500repeats block 516. In response to determining that the occurrence in thelive event has ended in the display of the live event, process 500proceeds to block 518.

At block 518, the control circuitry 304 generates for simultaneousdisplay, with the display of the live event, additional supplementalcontent 110 related to the occurrence. For example, in embodiments wherethe supplemental content 110 includes a poll, the control circuitry 304generates for display a result of the poll.

FIG. 6 is a flowchart of a detailed illustrative process for determininga real-time ending of an occurrence in a live event, in accordance withsome embodiments of the disclosure. A process 600 for determining areal-time ending of an occurrence in a live event may begin at block702.

At block 604, the control circuitry 304 retrieves metadata associatedwith the display of the live event. For example, the metadata may beembedded in the media content 104, or the control circuitry 304 mayretrieve the metadata separately from a database at the content source206.

At block 606, the control circuitry 304 determines whether the metadataincludes an indication of the ending of the occurrence in real time. Forexample, the control circuitry 304 may determine whether the metadataincludes timestamps or other synchronization data indicating that theoccurrence in the live event has ended in real time. In response todetermining that the metadata does not include an indication of theending of the occurrence in real time, process 600 ends. In response todetermining that the metadata includes an indication of the ending ofthe occurrence in real time, process 600 proceeds to block 608.

At block 608, the control circuitry 304 determines that the occurrencehas ended in real time based on the indication of the ending of theoccurrence included in the metadata. For example, the control circuitry304 may identify a timestamp indicating the ending of the occurrence inthe live event in real time, and may determine, based on the existenceof the timestamp, that the occurrence in the live event has ended inreal time.

FIG. 7 is a flowchart of a detailed illustrative process for determininga beginning of an occurrence in a display of a live event, in accordancewith some embodiments of the disclosure. A process 700 for determining abeginning of an occurrence in a display of a live event may begin atblock 702.

At block 704, the control circuitry 304 determines whether the displayof the live event is delayed. For example, the control circuitry 304 maydetermine whether the media content 104 is being played back from a livetransmission, or is time-shifted from the live transmission. In responseto determining that the display of the live event is not delayed, theprocess 700 ends. In response to determining that the display of thelive event is delayed, the process 700 proceeds to block 706.

At block 706, the control circuitry 304 determines an amount of delaybetween the live event and the display of the live event. For example,the control circuitry 304 may determine an amount of time that playbackof the media content 104 is behind the live transmission. Subprocessesdescribing further details on how the control circuitry 304 determinesthe amount of delay between the live event and the display of the liveevent are described below with reference to FIGS. 8 and 9 .

At block 708, the control circuitry 304 determines a start time of theoccurrence in the live event in real time. For example, the controlcircuitry 304 may determine a time when the field goal attempt startedin real time. In some embodiments, the control circuitry 304 retrievesmetadata associated with the media content 104 and identifies in themetadata a timestamp or other synchronization data indicating a starttime of the occurrence in the live event in real time.

At block 710, the control circuitry 304 calculates an offset start timeof the occurrence (i.e., the start time of the occurrence in the delayedplayback of the media content 104). In some embodiments, the controlcircuitry 304 calculates the offset start time by adding the amount ofdelay determined at block 706 to the start time of the occurrence in thelive event in real time determined at block 708.

FIG. 8 is a flowchart of a detailed illustrative process for determiningan amount of delay between a live event and a display of a live event,in accordance with some embodiments of the disclosure. A process 800 fordetermining an amount of delay between a live event and a display of thelive event may begin at block 802.

At block 804, the control circuitry 304 retrieves metadata associatedwith the display of the live event. For example, the metadata may beembedded in the media content 104, or the control circuitry 304 mayretrieve the metadata separately from a database at the content source206.

At block 806, the control circuitry determines whether the metadataincludes an indication of a time when a currently displayed portion ofthe display of the live event happened. For example, the controlcircuitry may identify, in the metadata, an indication (e.g., atimestamp or other synchronization data) of a time when a currentlydisplayed portion of the media content 104 happened in the live event inreal time. In response to determining that the metadata does not includean indication of the time when the currently displayed portion of thedisplay of the live event happened, the process 800 ends. In response todetermining that the metadata includes an indication of the time whenthe currently displayed portion of the display of the live eventhappened, the process 800 proceeds to block 808.

At block 808, the control circuitry 304 compares the time when thecurrently displayed portion of the live event happened with a currenttime. For example, the control circuitry may compare a timestampincluded in the metadata indicating the time when the currentlydisplayed portion of the media content 104 happened with a current time.

At block 810, the control circuitry determines whether the time when thecurrently displayed portion of the live event happened is different fromthe current time. In response to determining that the time when thecurrently displayed portion of the live event happened is not differentfrom the current time, process 800 ends. In response to determining thatthe time when the currently displayed portion of the live event happenedis different from the current time, process 800 proceeds to block 812.

At block 812, the control circuitry 304 calculates a difference betweenthe time when the currently displayed portion of the live event happenedand the current time. For example, the control circuitry 304 maysubtract the time when the currently displayed portion of the mediacontent 104 happened in real time from the current time, and the resultwill be the difference.

FIG. 9 is a flowchart of another detailed illustrative process fordetermining an amount of delay between a live event and a display of alive event, in accordance with some embodiments of the disclosure. Aprocess 900 for determining an amount of delay between a live event anda display of the live event may begin at block 902.

At block 904, the control circuitry 304 determines whether a user actionaffecting playback of the display of the live event has been performed.For example, the control circuitry 304 may determine whether playback ofthe media content 104 has been paused, rewound, fast-forwarded, etc. Inresponse to determining that a user action affecting playback of thedisplay of the live event has not been performed, process 900 ends. Inresponse to determining that a user action affecting playback of thedisplay of the live event has been performed, process 900 proceeds toblock 906.

At block 906, the control circuitry 304 calculates an amount oftime-shifting of the display of the live event has been caused by theuser action affecting playback of the display of the live event. Forexample, the control circuitry 304 may calculate for how long the userhas paused playback of the media content 104.

At block 908, the control circuitry 304 calculates a difference betweenthe live event and the currently displayed portion of the display basedon the amount of time-shifting. For example, if the control circuitry304 determines at block 906 that the user has paused playback of themedia content 104 for 30 seconds, the control circuitry may calculate atblock 908 that the difference (e.g., the delay) between the live eventand the currently displayed portion of the media content 104 is 30seconds.

FIG. 10 is a flowchart of a detailed illustrative process fordetermining a beginning of an occurrence in a display of a live event,in accordance with some embodiments of the disclosure. A process 1000for determining a beginning of an occurrence in a display of a liveevent may begin at block 1002.

At block 1004, the control circuitry 304 retrieves metadata associatedwith the display of the live event. For example, the metadata may beembedded in the media content 104, or the control circuitry 304 mayretrieve the metadata separately from a database at the content source206.

At block 1006, the control circuitry 304 determines whether the metadataincludes an indication of the beginning of the occurrence in the displayof the live event. For example, the control circuitry 304 may identify,in the metadata, an indication (e.g., a timestamp or othersynchronization data) of the time 422 when display of a field goalattempt in a football game begins in the delayed playback of the mediacontent 104. In response to determining that the metadata does notinclude an indication of the beginning of the occurrence in the displayof the live event, process 1000 ends. In response to determining thatthe metadata does include an indication of the beginning of theoccurrence in the display of the live event, the process 1000 proceedsto block 1008.

At block 1008, the control circuitry 304 determines the beginning of theoccurrence in the display of the live event based on the indication ofthe beginning of the occurrence included in the metadata. For example,the control circuitry 304 may determine that the time 422 corresponds tothe beginning of the field goal attempt based on a timestamp included inthe metadata.

FIG. 11 is a flowchart of another detailed illustrative process fordetermining a beginning of an occurrence in a display of a live event,in accordance with some embodiments of the disclosure. A process 1100for determining the beginning of the occurrence in the display of thelive event may begin at block 1102.

At block 1104, the control circuitry 304 identifies graphics or audioincluded in the display of the live event. For example, the controlcircuitry 304 may perform audio and/or image processing on the mediacontent 104 to identify graphics or audio included in the media content104 that indicate the beginning of the occurrence. In an embodiment, thecontrol circuitry 304 applies pattern recognition and/or characterrecognition techniques to the media content 104 to identify text,graphics, objects, and/or actions performed by objects in the mediacontent 104 that indicate the beginning of the occurrence.

At block 1106, the control circuitry 304 identifies a characteristicincluded in the graphics or audio identified at block 1104. For example,the control circuitry 304 may identify a gesture or sign displayed by areferee of a football game indicating that a field goal attempt is aboutto commence. The control circuitry 304 may also apply audio signatureanalysis and matching techniques to audio included in the media content104 to identify words indicating the beginning of the occurrence, oridentify sounds (e.g., cheers, whistles, etc.) indicating the beginningof the occurrence.

At block 1108, the control circuitry 304 determines whether thecharacteristic identified at block 1106 indicates a beginning of theoccurrence. In response to determining that the characteristic does notindicate the beginning of the occurrence, process 1100 returns to block1104 to identify additional graphics or audio included in the display ofthe live event. In response to determining that the characteristic doesindicate the beginning of the occurrence, process 1100 proceeds to block1110.

At block 1110, the control circuitry 304 determines the beginning of theoccurrence in the display of the live event based on the characteristic.For example, the control circuitry 304 may determine that a hand signaldisplayed by a referee of the football game indicates that a field goalattempt is about to begin.

FIG. 12 is a flowchart of a detailed illustrative process for generatingfor simultaneous display, with a display of a live event, supplementalcontent related to an occurrence, in accordance with some embodiments ofthe disclosure. A process 1200 for generating for simultaneous display,with the display of the live event, interactive content related to theoccurrence may begin at block 1202.

At block 1204, the control circuitry 304 generates a poll based oninformation related to the occurrence, such as the information retrievedat block 606. For example, the control circuitry 304 may generate a pollasking a user to predict and/or vote on whether the field goal attemptwill be successful. In some embodiments, the supplemental contentincludes an option to place a wager or bet on the outcome of theoccurrence. For example, the control circuitry 304 may generate awagering or betting option allowing the user to place a wager on whetherthe field goal attempt will be successful.

At block 1206, the control circuitry 304 transmits the poll to a userdevice, such as the second device 204. In some embodiments, thesupplemental content 110 is generated at, or transmitted to, the seconddevice 204.

At block 1208, the control circuitry 304 generates for display, on theuser device, the poll. For example, while generating for display themedia content 104 on the media consumption device 102, the controlcircuitry 304 may generate for display, on the second device 204, thepoll.

At block 1210, the control circuitry receives an indication of a userselection of a response to the poll. For example, the control circuitry304 may receive, such as via the user input interface 320, a userselection of a selectable option included in the supplemental content110. In some embodiments, the control circuitry 304 receives a userselection of a “yes” button indicating that the user predicts that thefield goal attempt will be successful. In some embodiments, the userselection of the selectable option is received via the second device204.

At block 1212, the control circuitry 304 determines a result of the pollbased on the indication of the user selection of the response to thepoll received at block 1210. For example, the control circuitry 304 mayreceive answers to the poll from multiple users via media consumptiondevices 102 and/or second devices 204, and may aggregate the answers todetermine a result of the poll.

At block 1214, the control circuitry 304 generates for display theresult of the poll. For example, the additional supplemental content 110generated for display at block 618 may include the result of the polldetermined at block 1212.

It should be noted that processes 500-1200 or any step thereof could beperformed on, or provided by, any of the devices shown in FIGS. 1-3 .For example, the processes may be executed by control circuitry 304(FIG. 3 ) as instructed by a content generation application implementedon media consumption device 102, content source 206, and/or processingserver 216. In addition, one or more steps of a process may be omitted,modified, and/or incorporated into or combined with one or more steps ofany other process or embodiment (e.g., steps from process 600 may becombined with steps from process 800). In addition, the steps anddescriptions described in relation to FIGS. 5-12 may be done inalternative orders or in parallel to further the purposes of thisdisclosure. For example, each of these steps may be performed in anyorder or in parallel or substantially simultaneously to reduce lag orincrease the speed of the system or method.

It will be apparent to those of ordinary skill in the art that methodsinvolved in the present invention may be embodied in a computer programproduct that includes a computer-usable and/or -readable medium. Forexample, such a computer-usable medium may consist of a read-only memorydevice, such as a CD-ROM disk or conventional ROM device, or arandom-access memory, such as a hard drive device or a computerdiskette, having a computer-readable program code stored thereon. Itshould also be understood that methods, techniques, and processesinvolved in the present disclosure may be executed using processingcircuitry.

The processes discussed above are intended to be illustrative and notlimiting. More generally, the above disclosure is meant to be exemplaryand not limiting. Only the claims that follow are meant to set bounds asto what the present invention includes. Furthermore, it should be notedthat the features and limitations described in any one embodiment may beapplied to any other embodiment herein, and flowcharts or examplesrelating to one embodiment may be combined with any other embodiment ina suitable manner, done in different orders, or done in parallel. Inaddition, the systems and methods described herein may be performed inreal time. It should also be noted, the systems and/or methods describedabove may be applied to, or used in accordance with, other systemsand/or methods.

1-50. (canceled)
 51. A method comprising: receiving a stream of mediacontent, wherein the media content corresponds to a live event;processing the media content to identify metadata indicative of abeginning of a live event in the media content; in response toidentifying the metadata indicative of the beginning of the live eventin the media content: generating for presentation a user interfaceelement comprising a prompt to input a prediction associated with thelive event; and determining whether the live event has ended inreal-time; and in response to determining that the live event has endedin real-time, cease generating for presentation the user interfaceelement.
 52. The method of claim 51, wherein processing the mediacontent to identify the metadata indicative of the beginning of the liveevent in the media content comprises: performing audio processing toidentify audio features of the media content indicative of the beginningof the live event.
 53. The method of claim 51, wherein processing themedia content to identify the metadata indicative of the beginning ofthe live event in the media content comprises: performing imageprocessing to identify graphics, objects, or text of the media contentindicative of the beginning of the live event.
 54. The method of claim51, wherein the media content is delayed relative to the live event. 55.The method of claim 51, wherein the metadata comprises a keywordindicating the live event has begun.
 56. The method of claim 51, whereinthe metadata comprises a timestamp indicating a real-time time point atwhich the live event began.
 57. The method of claim 51, wherein themetadata is embedded in the media content or is retrieved separatelyfrom the media content.
 58. The method of claim 51, wherein generatingfor presentation the user interface element comprising the prompt toinput the prediction associated with the live event comprises:retrieving information related to the live event; and generating theuser interface element based on the information related to the liveevent, wherein the prompt includes statistics related to a predictedoutcome of the live event.
 59. The method of claim 58, furthercomprising: receiving, from each of a plurality of user devices, acorresponding prediction associated with the live event; aggregating thepredictions received from the plurality of user devices; and in responseto determining that the live event has ended in real-time, generatingfor presentation statistics related to the predictions received from theplurality of user devices.
 60. The method of claim 51, wherein the mediacontent is generated for presentation on a first device, and the userinterface element comprising the prompt is generated for presentation ona second device that is different from the first device.
 61. A systemcomprising: memory; and control circuitry configured to: receive astream of media content, wherein the media content corresponds to a liveevent; process the media content to identify metadata indicative of abeginning of a live event in the media content, wherein the metadata isstored in memory; in response to identifying the metadata indicative ofthe beginning of the live event in the media content: generate forpresentation a user interface element comprising a prompt to input aprediction associated with the live event; and determine whether thelive event has ended in real-time; and in response to determining thatthe live event has ended in real-time, cease generating for presentationthe user interface element.
 62. The system of claim 61, wherein thecontrol circuitry is configured to process the media content to identifythe metadata indicative of the beginning of the live event in the mediacontent by: performing audio processing to identify audio features ofthe media content indicative of the beginning of the live event.
 63. Thesystem of claim 61, wherein the control circuitry is configured toprocess the media content to identify the metadata indicative of thebeginning of the live event in the media content by: performing imageprocessing to identify graphics, objects, or text of the media contentindicative of the beginning of the live event.
 64. The system of claim61, wherein the media content is delayed relative to the live event. 65.The system of claim 61, wherein the metadata comprises a keywordindicating the live event has begun.
 66. The system of claim 61, whereinthe metadata comprises a timestamp indicating a real-time time point atwhich the live event began.
 67. The system of claim 61, wherein themetadata is embedded in the media content or is retrieved separatelyfrom the media content.
 68. The system of claim 61, wherein the controlcircuitry is configured to generate for presentation the user interfaceelement comprising the prompt to input the prediction associated withthe live event by: retrieving information related to the live event; andgenerating the user interface element based on the information relatedto the live event, wherein the prompt includes statistics related to apredicted outcome of the live event.
 69. The system of claim 68, whereinthe control circuitry is further configured to: receive, from each of aplurality of user devices, a corresponding prediction associated withthe live event; aggregate the predictions received from the plurality ofuser devices; and in response to determining that the live event hasended in real-time, generate for presentation statistics related to thepredictions received from the plurality of user devices.
 70. The systemof claim 61, wherein the media content is generated for presentation ona first device, and the user interface element comprising the prompt isgenerated for presentation on a second device that is different from thefirst device.